{"cells":[{"source":"<a href=\"https://www.kaggle.com/code/peremartramanonellas/rouge-evaluation-untrained-vs-trained-llm?scriptVersionId=139928805\" target=\"_blank\"><img align=\"left\" alt=\"Kaggle\" title=\"Open in Kaggle\" src=\"https://kaggle.com/static/images/open-in-kaggle.svg\"></a>","metadata":{},"cell_type":"markdown"},{"cell_type":"code","execution_count":1,"id":"4a3e7bd2","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.execute_input":"2023-08-14T22:02:21.353196Z","iopub.status.busy":"2023-08-14T22:02:21.352498Z","iopub.status.idle":"2023-08-14T22:02:21.367953Z","shell.execute_reply":"2023-08-14T22:02:21.366828Z"},"papermill":{"duration":0.03269,"end_time":"2023-08-14T22:02:21.370334","exception":false,"start_time":"2023-08-14T22:02:21.337644","status":"completed"},"tags":[]},"outputs":[{"data":{"text/html":["\n","<style>\n",".output_png {\n","    display: table-cell;\n","    text-align: center;\n","    vertical-align: middle;\n","    horizontal-align: middle;\n","}\n","h1 {\n","    text-align: center;\n","    background-color: #6bacf5;\n","    padding: 10px;\n","    margin: 0;\n","    font-family: monospace;\n","    color:DimGray;\n","    border-radius: 2px\n","    style=\"font-family:verdana;\"\n","}\n","\n","h2 {\n","    text-align: center;\n","    background-color: #83c2ff;\n","    padding: 10px;\n","    margin: 0;\n","    font-family: monospace;\n","    color:DimGray;\n","    border-radius: 2px\n","}\n","\n","h3 {\n","    text-align: center;\n","    background-color: pink;\n","    padding: 10px;\n","    margin: 0;\n","    font-family: monospace;\n","    color:DimGray;\n","    border-radius: 2px\n","}\n","\n","h4 {\n","    text-align: center;\n","    background-color: pink;\n","    padding: 10px;\n","    margin: 0;\n","    font-family: monospace;\n","    color:DimGray;\n","    border-radius: 2px\n","}\n","\n","body, p {\n","    font-family: monospace;\n","    font-size: 18px;\n","    color: charcoal;\n","}\n","div {\n","    font-size: 14px;\n","    margin: 0;\n","\n","}\n","\n","\n","</style>\n"],"text/plain":["<IPython.core.display.HTML object>"]},"execution_count":1,"metadata":{},"output_type":"execute_result"}],"source":["from IPython.core.display import HTML\n","HTML(\"\"\"\n","<style>\n",".output_png {\n","    display: table-cell;\n","    text-align: center;\n","    vertical-align: middle;\n","    horizontal-align: middle;\n","}\n","h1 {\n","    text-align: center;\n","    background-color: #6bacf5;\n","    padding: 10px;\n","    margin: 0;\n","    font-family: monospace;\n","    color:DimGray;\n","    border-radius: 2px\n","    style=\"font-family:verdana;\"\n","}\n","\n","h2 {\n","    text-align: center;\n","    background-color: #83c2ff;\n","    padding: 10px;\n","    margin: 0;\n","    font-family: monospace;\n","    color:DimGray;\n","    border-radius: 2px\n","}\n","\n","h3 {\n","    text-align: center;\n","    background-color: pink;\n","    padding: 10px;\n","    margin: 0;\n","    font-family: monospace;\n","    color:DimGray;\n","    border-radius: 2px\n","}\n","\n","h4 {\n","    text-align: center;\n","    background-color: pink;\n","    padding: 10px;\n","    margin: 0;\n","    font-family: monospace;\n","    color:DimGray;\n","    border-radius: 2px\n","}\n","\n","body, p {\n","    font-family: monospace;\n","    font-size: 18px;\n","    color: charcoal;\n","}\n","div {\n","    font-size: 14px;\n","    margin: 0;\n","\n","}\n","\n","\n","</style>\n","\"\"\")"]},{"cell_type":"markdown","id":"2feaa00c","metadata":{"papermill":{"duration":0.012459,"end_time":"2023-08-14T22:02:21.395806","exception":false,"start_time":"2023-08-14T22:02:21.383347","status":"completed"},"tags":[]},"source":["# How to Evaluate Large Language Models for Summarization Using ROUGE.\n","The way we evaluate large language models is quite different from evaluating machine learning models, where metrics like Accuracy, F1 Score, or Recall were commonly used. \n","\n","Metrics for generated language are distinct. Depending on the specific application, different metrics are chosen to assess the model's performance. \n","\n","In this notebook, we will explore the usage of the ROUGE metric to measure the quality of summaries generated by a language model.\n","\n","### This notebook is part of a comprehensive course on Large Language Models available on GitHub: https://github.com/peremartra/Large-Language-Model-Notebooks-Course. If you want to stay informed about new lessons or updates, simply follow or star the repository.\n","\n","## What is ROUGE?\n","ROUGE isn't just a single metric; it's a set of metrics that measure the overlap and similarity between the generated summary and a reference summary that serves as a benchmark.\n","\n","It returns fourth individual metrics. The metrics provided are:\n","\n","* ROUGE-1: Measures the overlap of unigrams, or single words.\n","* ROUGE-2: Measures the overlap of bigrams, or pairs of words.\n","* ROUGE-L: Measures the longest common subsequence, rewarding longer shared sequences between the generated and reference summaries.\n","* ROUGE-LSUM: Calculated as the length of the LCS divided by the sum of the lengths of the generated summary and the reference summary. \n","\n","### Feel Free to fork or edit the noteboook for you own convenience. Please consider ***UPVOTING IT***. It helps others to discover the notebook, and it encourages me to continue publishing.\n","\n","## What are we going to do? \n","We are going to use two T5 models, one of them being the t5-Base model and the other a t5-base fine-tuned  specifically designed for creating summaries. \n","\n","First, we will use a dataset and generate summaries using both models. By comparing the two generated summaries, we can observe whether the fine-tuning has been effective in producing different results. In other words, here we will only determine that the two models exhibit significant differences in summary generation, but we won't know which one might perform better.\n","\n","To determine which model generates better summaries, we will utilize a well-known dataset called 'cnn_dailymail,' which is available in the 'datasets' library. \n","\n","This dataset contains reference summaries that can be used for comparison. We will assess the summaries generated by the two models against these reference summaries.\n","\n","The model that obtains a higher ROUGE score will be considered the one that produces better summaries.\n","\n","## The models.\n","t5-Base Finnetunned: https://huggingface.co/flax-community/t5-base-cnn-\n","\n","t5-Base: https://huggingface.co/t5-base\n"]},{"cell_type":"markdown","id":"6466b7f5","metadata":{"papermill":{"duration":0.013868,"end_time":"2023-08-14T22:02:21.4225","exception":false,"start_time":"2023-08-14T22:02:21.408632","status":"completed"},"tags":[]},"source":["# Load the Data"]},{"cell_type":"code","execution_count":2,"id":"c1bfad0c","metadata":{"execution":{"iopub.execute_input":"2023-08-14T22:02:21.450466Z","iopub.status.busy":"2023-08-14T22:02:21.449273Z","iopub.status.idle":"2023-08-14T22:02:25.012019Z","shell.execute_reply":"2023-08-14T22:02:25.011006Z"},"papermill":{"duration":3.579718,"end_time":"2023-08-14T22:02:25.015059","exception":false,"start_time":"2023-08-14T22:02:21.435341","status":"completed"},"tags":[]},"outputs":[],"source":["#Import generic libraries\n","import numpy as np \n","import pandas as pd\n","import torch\n"]},{"cell_type":"markdown","id":"9463e9f0","metadata":{"papermill":{"duration":0.012577,"end_time":"2023-08-14T22:02:25.040898","exception":false,"start_time":"2023-08-14T22:02:25.028321","status":"completed"},"tags":[]},"source":["The dataset is available on Kaggle and comprises a collection of technological news articles compiled by MIT. The article text is located in the 'Article Body' column.\n","\n","https://www.kaggle.com/datasets/deepanshudalal09/mit-ai-news-published-till-2023"]},{"cell_type":"code","execution_count":3,"id":"fed7edd6","metadata":{"execution":{"iopub.execute_input":"2023-08-14T22:02:25.068957Z","iopub.status.busy":"2023-08-14T22:02:25.06828Z","iopub.status.idle":"2023-08-14T22:02:25.270255Z","shell.execute_reply":"2023-08-14T22:02:25.269362Z"},"papermill":{"duration":0.219225,"end_time":"2023-08-14T22:02:25.273176","exception":false,"start_time":"2023-08-14T22:02:25.053951","status":"completed"},"tags":[]},"outputs":[],"source":["news = pd.read_csv('/kaggle/input/mit-ai-news-published-till-2023/articles.csv')\n","DOCUMENT=\"Article Body\""]},{"cell_type":"code","execution_count":4,"id":"e13d6679","metadata":{"execution":{"iopub.execute_input":"2023-08-14T22:02:25.305575Z","iopub.status.busy":"2023-08-14T22:02:25.304951Z","iopub.status.idle":"2023-08-14T22:02:25.310307Z","shell.execute_reply":"2023-08-14T22:02:25.309257Z"},"papermill":{"duration":0.022404,"end_time":"2023-08-14T22:02:25.313066","exception":false,"start_time":"2023-08-14T22:02:25.290662","status":"completed"},"tags":[]},"outputs":[],"source":["#Because it is just a course we select a small portion of News.\n","MAX_NEWS = 3\n","subset_news = news.head(MAX_NEWS)"]},{"cell_type":"code","execution_count":5,"id":"d0ebc0be","metadata":{"execution":{"iopub.execute_input":"2023-08-14T22:02:25.346059Z","iopub.status.busy":"2023-08-14T22:02:25.344835Z","iopub.status.idle":"2023-08-14T22:02:25.368428Z","shell.execute_reply":"2023-08-14T22:02:25.367384Z"},"papermill":{"duration":0.04048,"end_time":"2023-08-14T22:02:25.370798","exception":false,"start_time":"2023-08-14T22:02:25.330318","status":"completed"},"tags":[]},"outputs":[{"data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>Unnamed: 0</th>\n","      <th>Published Date</th>\n","      <th>Author</th>\n","      <th>Source</th>\n","      <th>Article Header</th>\n","      <th>Sub_Headings</th>\n","      <th>Article Body</th>\n","      <th>Url</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>0</td>\n","      <td>July 7, 2023</td>\n","      <td>Adam Zewe</td>\n","      <td>MIT News Office</td>\n","      <td>Learning the language of molecules to predict ...</td>\n","      <td>This AI system only needs a small amount of da...</td>\n","      <td>['Discovering new materials and drugs typicall...</td>\n","      <td>https://news.mit.edu/2023/learning-language-mo...</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>1</td>\n","      <td>July 6, 2023</td>\n","      <td>Alex Ouyang</td>\n","      <td>Abdul Latif Jameel Clinic for Machine Learning...</td>\n","      <td>MIT scientists build a system that can generat...</td>\n","      <td>BioAutoMATED, an open-source, automated machin...</td>\n","      <td>['Is it possible to build machine-learning mod...</td>\n","      <td>https://news.mit.edu/2023/bioautomated-open-so...</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>2</td>\n","      <td>June 30, 2023</td>\n","      <td>Jennifer Michalowski</td>\n","      <td>McGovern Institute for Brain Research</td>\n","      <td>When computer vision works more like a brain, ...</td>\n","      <td>Training artificial neural networks with data ...</td>\n","      <td>['From cameras to self-driving cars, many of t...</td>\n","      <td>https://news.mit.edu/2023/when-computer-vision...</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["   Unnamed: 0 Published Date                Author  \\\n","0           0   July 7, 2023             Adam Zewe   \n","1           1   July 6, 2023           Alex Ouyang   \n","2           2  June 30, 2023  Jennifer Michalowski   \n","\n","                                              Source  \\\n","0                                    MIT News Office   \n","1  Abdul Latif Jameel Clinic for Machine Learning...   \n","2              McGovern Institute for Brain Research   \n","\n","                                      Article Header  \\\n","0  Learning the language of molecules to predict ...   \n","1  MIT scientists build a system that can generat...   \n","2  When computer vision works more like a brain, ...   \n","\n","                                        Sub_Headings  \\\n","0  This AI system only needs a small amount of da...   \n","1  BioAutoMATED, an open-source, automated machin...   \n","2  Training artificial neural networks with data ...   \n","\n","                                        Article Body  \\\n","0  ['Discovering new materials and drugs typicall...   \n","1  ['Is it possible to build machine-learning mod...   \n","2  ['From cameras to self-driving cars, many of t...   \n","\n","                                                 Url  \n","0  https://news.mit.edu/2023/learning-language-mo...  \n","1  https://news.mit.edu/2023/bioautomated-open-so...  \n","2  https://news.mit.edu/2023/when-computer-vision...  "]},"execution_count":5,"metadata":{},"output_type":"execute_result"}],"source":["subset_news.head()"]},{"cell_type":"code","execution_count":6,"id":"ad7e9329","metadata":{"execution":{"iopub.execute_input":"2023-08-14T22:02:25.400413Z","iopub.status.busy":"2023-08-14T22:02:25.399937Z","iopub.status.idle":"2023-08-14T22:02:25.405688Z","shell.execute_reply":"2023-08-14T22:02:25.404595Z"},"papermill":{"duration":0.024357,"end_time":"2023-08-14T22:02:25.40825","exception":false,"start_time":"2023-08-14T22:02:25.383893","status":"completed"},"tags":[]},"outputs":[],"source":["articles = subset_news[DOCUMENT].tolist()"]},{"cell_type":"markdown","id":"359993aa","metadata":{"papermill":{"duration":0.013052,"end_time":"2023-08-14T22:02:25.434756","exception":false,"start_time":"2023-08-14T22:02:25.421704","status":"completed"},"tags":[]},"source":["# Load the Models and create the summaries\n","\n","Both models are available on Hugging Face, so we will work with the Transformers library."]},{"cell_type":"code","execution_count":7,"id":"906947a6","metadata":{"execution":{"iopub.execute_input":"2023-08-14T22:02:25.463399Z","iopub.status.busy":"2023-08-14T22:02:25.462666Z","iopub.status.idle":"2023-08-14T22:02:27.592118Z","shell.execute_reply":"2023-08-14T22:02:27.59115Z"},"papermill":{"duration":2.146867,"end_time":"2023-08-14T22:02:27.594898","exception":false,"start_time":"2023-08-14T22:02:25.448031","status":"completed"},"tags":[]},"outputs":[],"source":["import transformers\n","from transformers import AutoTokenizer, AutoModelForSeq2SeqLM\n","\n","model_name_small = \"t5-base\"\n","model_name_reference = \"flax-community/t5-base-cnn-dm\"\n","#model_name_reference = \"pszemraj/long-t5-tglobal-base-16384-booksum-V11-big_patent-V2\""]},{"cell_type":"code","execution_count":8,"id":"544b7159","metadata":{"execution":{"iopub.execute_input":"2023-08-14T22:02:27.623394Z","iopub.status.busy":"2023-08-14T22:02:27.622565Z","iopub.status.idle":"2023-08-14T22:02:27.627878Z","shell.execute_reply":"2023-08-14T22:02:27.627035Z"},"papermill":{"duration":0.022031,"end_time":"2023-08-14T22:02:27.630092","exception":false,"start_time":"2023-08-14T22:02:27.608061","status":"completed"},"tags":[]},"outputs":[],"source":["#This function returns the tokenizer and the Model. \n","def get_model(model_id):\n","    tokenizer = AutoTokenizer.from_pretrained(model_id)\n","    model = AutoModelForSeq2SeqLM.from_pretrained(model_id)\n","    \n","    return tokenizer, model\n","    "]},{"cell_type":"code","execution_count":9,"id":"fbbaa67f","metadata":{"execution":{"iopub.execute_input":"2023-08-14T22:02:27.658002Z","iopub.status.busy":"2023-08-14T22:02:27.657508Z","iopub.status.idle":"2023-08-14T22:02:48.180447Z","shell.execute_reply":"2023-08-14T22:02:48.178839Z"},"papermill":{"duration":20.540525,"end_time":"2023-08-14T22:02:48.183564","exception":false,"start_time":"2023-08-14T22:02:27.643039","status":"completed"},"tags":[]},"outputs":[{"data":{"application/vnd.jupyter.widget-view+json":{"model_id":"4b20dfdba30342078e55ec3aaba678f6","version_major":2,"version_minor":0},"text/plain":["Downloading (…)lve/main/config.json:   0%|          | 0.00/1.21k [00:00<?, ?B/s]"]},"metadata":{},"output_type":"display_data"},{"data":{"application/vnd.jupyter.widget-view+json":{"model_id":"6c8aca16f03b4f85bd431d3e701beca9","version_major":2,"version_minor":0},"text/plain":["Downloading (…)ve/main/spiece.model:   0%|          | 0.00/792k [00:00<?, ?B/s]"]},"metadata":{},"output_type":"display_data"},{"data":{"application/vnd.jupyter.widget-view+json":{"model_id":"f7fb4e3a04be4b31a36924431df8f33a","version_major":2,"version_minor":0},"text/plain":["Downloading (…)/main/tokenizer.json:   0%|          | 0.00/1.39M [00:00<?, ?B/s]"]},"metadata":{},"output_type":"display_data"},{"name":"stderr","output_type":"stream","text":["/opt/conda/lib/python3.10/site-packages/transformers/models/t5/tokenization_t5_fast.py:155: FutureWarning: This tokenizer was incorrectly instantiated with a model max length of 512 which will be corrected in Transformers v5.\n","For now, this behavior is kept to avoid breaking backwards compatibility when padding/encoding with `truncation is True`.\n","- Be aware that you SHOULD NOT rely on t5-base automatically truncating your input to 512 when padding/encoding.\n","- If you want to encode/pad to sequences longer than 512 you can either instantiate this tokenizer with `model_max_length` or pass `max_length` when encoding/padding.\n","- To avoid this warning, please instantiate this tokenizer with `model_max_length` set to your preferred value.\n","  warnings.warn(\n","/opt/conda/lib/python3.10/site-packages/scipy/__init__.py:146: UserWarning: A NumPy version >=1.16.5 and <1.23.0 is required for this version of SciPy (detected version 1.23.5\n","  warnings.warn(f\"A NumPy version >={np_minversion} and <{np_maxversion}\"\n","/opt/conda/lib/python3.10/site-packages/tensorflow_io/python/ops/__init__.py:98: UserWarning: unable to load libtensorflow_io_plugins.so: unable to open file: libtensorflow_io_plugins.so, from paths: ['/opt/conda/lib/python3.10/site-packages/tensorflow_io/python/ops/libtensorflow_io_plugins.so']\n","caused by: ['/opt/conda/lib/python3.10/site-packages/tensorflow_io/python/ops/libtensorflow_io_plugins.so: undefined symbol: _ZN3tsl6StatusC1EN10tensorflow5error4CodeESt17basic_string_viewIcSt11char_traitsIcEENS_14SourceLocationE']\n","  warnings.warn(f\"unable to load libtensorflow_io_plugins.so: {e}\")\n","/opt/conda/lib/python3.10/site-packages/tensorflow_io/python/ops/__init__.py:104: UserWarning: file system plugins are not loaded: unable to open file: libtensorflow_io.so, from paths: ['/opt/conda/lib/python3.10/site-packages/tensorflow_io/python/ops/libtensorflow_io.so']\n","caused by: ['/opt/conda/lib/python3.10/site-packages/tensorflow_io/python/ops/libtensorflow_io.so: undefined symbol: _ZTVN10tensorflow13GcsFileSystemE']\n","  warnings.warn(f\"file system plugins are not loaded: {e}\")\n"]},{"data":{"application/vnd.jupyter.widget-view+json":{"model_id":"4246a421c6204b91a0b3f3d8b24753a9","version_major":2,"version_minor":0},"text/plain":["Downloading model.safetensors:   0%|          | 0.00/892M [00:00<?, ?B/s]"]},"metadata":{},"output_type":"display_data"},{"data":{"application/vnd.jupyter.widget-view+json":{"model_id":"afd407804c024cb6b1029e8ebad70f73","version_major":2,"version_minor":0},"text/plain":["Downloading (…)neration_config.json:   0%|          | 0.00/147 [00:00<?, ?B/s]"]},"metadata":{},"output_type":"display_data"}],"source":["tokenizer_small, model_small = get_model(model_name_small)"]},{"cell_type":"code","execution_count":10,"id":"7934f671","metadata":{"execution":{"iopub.execute_input":"2023-08-14T22:02:48.215342Z","iopub.status.busy":"2023-08-14T22:02:48.214347Z","iopub.status.idle":"2023-08-14T22:03:35.738349Z","shell.execute_reply":"2023-08-14T22:03:35.736901Z"},"papermill":{"duration":47.542959,"end_time":"2023-08-14T22:03:35.74126","exception":false,"start_time":"2023-08-14T22:02:48.198301","status":"completed"},"tags":[]},"outputs":[{"data":{"application/vnd.jupyter.widget-view+json":{"model_id":"9cd2db54e977486d9c1b0f5ef5032a30","version_major":2,"version_minor":0},"text/plain":["Downloading (…)okenizer_config.json:   0%|          | 0.00/1.92k [00:00<?, ?B/s]"]},"metadata":{},"output_type":"display_data"},{"data":{"application/vnd.jupyter.widget-view+json":{"model_id":"4c4b065791d54410b4898e983e32e568","version_major":2,"version_minor":0},"text/plain":["Downloading (…)/main/tokenizer.json:   0%|          | 0.00/1.39M [00:00<?, ?B/s]"]},"metadata":{},"output_type":"display_data"},{"data":{"application/vnd.jupyter.widget-view+json":{"model_id":"1ff74ca8cb38452b8145dda95757a0fb","version_major":2,"version_minor":0},"text/plain":["Downloading (…)cial_tokens_map.json:   0%|          | 0.00/1.79k [00:00<?, ?B/s]"]},"metadata":{},"output_type":"display_data"},{"data":{"application/vnd.jupyter.widget-view+json":{"model_id":"2b2463a361b24402ad7834c48ff13781","version_major":2,"version_minor":0},"text/plain":["Downloading (…)lve/main/config.json:   0%|          | 0.00/1.36k [00:00<?, ?B/s]"]},"metadata":{},"output_type":"display_data"},{"data":{"application/vnd.jupyter.widget-view+json":{"model_id":"1c6424e9bb1f44589a3b6153f7d8aaf1","version_major":2,"version_minor":0},"text/plain":["Downloading model.safetensors:   0%|          | 0.00/892M [00:00<?, ?B/s]"]},"metadata":{},"output_type":"display_data"}],"source":["tokenizer_reference, model_reference = get_model(model_name_reference)"]},{"cell_type":"markdown","id":"a423d166","metadata":{"papermill":{"duration":0.015182,"end_time":"2023-08-14T22:03:35.772039","exception":false,"start_time":"2023-08-14T22:03:35.756857","status":"completed"},"tags":[]},"source":["With both models downloaded and ready, we create a function that will perform the summaries.\n","\n","The function takes fourth parameters:\n","\n","* the list of texts to summarize.\n","* the tokenizer.\n","* the model.\n","* the maximum length for the generated summary"]},{"cell_type":"code","execution_count":11,"id":"7fc6d899","metadata":{"execution":{"iopub.execute_input":"2023-08-14T22:03:35.805553Z","iopub.status.busy":"2023-08-14T22:03:35.804721Z","iopub.status.idle":"2023-08-14T22:03:35.813463Z","shell.execute_reply":"2023-08-14T22:03:35.812706Z"},"papermill":{"duration":0.028015,"end_time":"2023-08-14T22:03:35.815725","exception":false,"start_time":"2023-08-14T22:03:35.78771","status":"completed"},"tags":[]},"outputs":[],"source":["def create_summaries(texts_list, tokenizer, model, max_l=125):\n","    \n","    # We are going to add a prefix to each article to be summarized \n","    # so that the model knows what it should do\n","    prefix = \"Summarize this news: \"  \n","    summaries_list = [] #Will contain all summaries\n","\n","    texts_list = [prefix + text for text in texts_list]\n","    \n","    for text in texts_list:\n","        \n","        summary=\"\"\n","        \n","        #calculate the encodings\n","        input_encodings = tokenizer(text, \n","                                    max_length=1024, \n","                                    return_tensors='pt', \n","                                    padding=True, \n","                                    truncation=True)\n","\n","        # Generate summaries\n","        with torch.no_grad():\n","            output = model.generate(\n","                input_ids=input_encodings.input_ids,\n","                attention_mask=input_encodings.attention_mask,\n","                max_length=max_l,  # Set the maximum length of the generated summary\n","                num_beams=2,     # Set the number of beams for beam search\n","                early_stopping=True\n","            )\n","            \n","        #Decode to get the text\n","        summary = tokenizer.batch_decode(output, skip_special_tokens=True)\n","        \n","        #Add the summary to summaries list \n","        summaries_list += summary\n","    return summaries_list \n","    "]},{"cell_type":"markdown","id":"4ffdb4fa","metadata":{"papermill":{"duration":0.015207,"end_time":"2023-08-14T22:03:35.84634","exception":false,"start_time":"2023-08-14T22:03:35.831133","status":"completed"},"tags":[]},"source":["To create the summaries, we call the 'create_summaries' function, passing both the news articles and the corresponding tokenizer and model."]},{"cell_type":"code","execution_count":12,"id":"ddd00e31","metadata":{"execution":{"iopub.execute_input":"2023-08-14T22:03:35.879443Z","iopub.status.busy":"2023-08-14T22:03:35.878646Z","iopub.status.idle":"2023-08-14T22:04:05.971552Z","shell.execute_reply":"2023-08-14T22:04:05.970547Z"},"papermill":{"duration":30.112965,"end_time":"2023-08-14T22:04:05.974752","exception":false,"start_time":"2023-08-14T22:03:35.861787","status":"completed"},"tags":[]},"outputs":[],"source":["# Creating the summaries for both models. \n","summaries_small = create_summaries(articles, \n","                                  tokenizer_small, \n","                                  model_small)\n"]},{"cell_type":"code","execution_count":13,"id":"2002eec6","metadata":{"execution":{"iopub.execute_input":"2023-08-14T22:04:06.009738Z","iopub.status.busy":"2023-08-14T22:04:06.008788Z","iopub.status.idle":"2023-08-14T22:04:37.085514Z","shell.execute_reply":"2023-08-14T22:04:37.084489Z"},"papermill":{"duration":31.095807,"end_time":"2023-08-14T22:04:37.088051","exception":false,"start_time":"2023-08-14T22:04:05.992244","status":"completed"},"tags":[]},"outputs":[],"source":["summaries_reference = create_summaries(articles, \n","                                      tokenizer_reference, \n","                                      model_reference)"]},{"cell_type":"code","execution_count":14,"id":"df297d1b","metadata":{"execution":{"iopub.execute_input":"2023-08-14T22:04:37.120847Z","iopub.status.busy":"2023-08-14T22:04:37.120424Z","iopub.status.idle":"2023-08-14T22:04:37.127523Z","shell.execute_reply":"2023-08-14T22:04:37.126412Z"},"papermill":{"duration":0.026332,"end_time":"2023-08-14T22:04:37.129872","exception":false,"start_time":"2023-08-14T22:04:37.10354","status":"completed"},"tags":[]},"outputs":[{"data":{"text/plain":["['MIT and MIT-Watson AI Lab have developed a unified framework. the system can simultaneously predict molecular properties and generate new molecules. it uses this grammar to construct viable molecules and predict their properties.',\n"," '\\'BioAutoMATED\\' is an automated machine-learning system that can select and build an appropriate model for a given dataset. it can even take care of the laborious task of data preprocessing, whittling down a months-long process to just a few hours. \\'\"We want to lower these barriers for a lot of folks that want to use machine learning or biology,\" says first co-author Jacqueline Valeri.',\n"," \"MIT and IBM research scientists have made a computer vision model more robust by training it to work like a part of the brain that humans and other primates rely on for object recognition. 'we asked the artificial neural network to make the function of one of your inside simulated “neural” layers as similar as possible to the corresponding biological neural layer,' says MIT professor.\"]"]},"execution_count":14,"metadata":{},"output_type":"execute_result"}],"source":["summaries_small"]},{"cell_type":"code","execution_count":15,"id":"887bcc1a","metadata":{"execution":{"iopub.execute_input":"2023-08-14T22:04:37.163043Z","iopub.status.busy":"2023-08-14T22:04:37.16222Z","iopub.status.idle":"2023-08-14T22:04:37.169083Z","shell.execute_reply":"2023-08-14T22:04:37.168144Z"},"papermill":{"duration":0.026038,"end_time":"2023-08-14T22:04:37.171289","exception":false,"start_time":"2023-08-14T22:04:37.145251","status":"completed"},"tags":[]},"outputs":[{"data":{"text/plain":["['Researchers created a machine-learning system that automatically learns the \"language\" of molecules using only a small, domain-specific dataset. The system learns to construct viable molecules and predict their properties. Computational design and Fabrication Group will be presented at the International Conference for Machine Learning.',\n"," \"Automated machine-learning system can select and build an appropriate model for a given dataset. 'BioAutoMATED' is an automated machine-learning system. The tool includes binary classification models, multi-class classification models, and more complex neural networks.\",\n"," \"MIT and IBM researchers have found that artificial neural networks resemble the multilayered brain circuits that process visual information in humans and other primates. 'We asked it to do both of those things as well as the standard, computer vision approach,' said one expert. The network found to be more robust by training it to work like a part of the brain that humans rely on for object recognition.\"]"]},"execution_count":15,"metadata":{},"output_type":"execute_result"}],"source":["summaries_reference"]},{"cell_type":"markdown","id":"5feaed8e","metadata":{"papermill":{"duration":0.015366,"end_time":"2023-08-14T22:04:37.20224","exception":false,"start_time":"2023-08-14T22:04:37.186874","status":"completed"},"tags":[]},"source":["At first glance, it's evident that the summaries are different. \n","\n","However, it's challenging to determine which one is better. \n","\n","It's even difficult to discern whether they are significantly distinct or if there are just subtle differences between them.\n","\n","This is what we are going to verify now using ROUGE. When comparing the summaries of one model with those of the other, we don't get an idea of which one is better, but rather an idea of how much the summaries have changed with the fine-tuning applied to the model."]},{"cell_type":"markdown","id":"6d77e02c","metadata":{"papermill":{"duration":0.015293,"end_time":"2023-08-14T22:04:37.23328","exception":false,"start_time":"2023-08-14T22:04:37.217987","status":"completed"},"tags":[]},"source":["# ROUGE\n","Let's install and load all the necessary libraries to conduct a ROUGE evaluation."]},{"cell_type":"code","execution_count":16,"id":"0b168dda","metadata":{"execution":{"iopub.execute_input":"2023-08-14T22:04:37.266661Z","iopub.status.busy":"2023-08-14T22:04:37.265998Z","iopub.status.idle":"2023-08-14T22:04:38.174437Z","shell.execute_reply":"2023-08-14T22:04:38.172577Z"},"papermill":{"duration":0.9274,"end_time":"2023-08-14T22:04:38.17621","exception":true,"start_time":"2023-08-14T22:04:37.24881","status":"failed"},"tags":[]},"outputs":[{"data":{"text/html":["<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #800000; text-decoration-color: #800000\">╭─────────────────────────────── </span><span style=\"color: #800000; text-decoration-color: #800000; font-weight: bold\">Traceback </span><span style=\"color: #bf7f7f; text-decoration-color: #bf7f7f; font-weight: bold\">(most recent call last)</span><span style=\"color: #800000; text-decoration-color: #800000\"> ────────────────────────────────╮</span>\n","<span style=\"color: #800000; text-decoration-color: #800000\">│</span> in <span style=\"color: #00ff00; text-decoration-color: #00ff00\">&lt;module&gt;</span>:<span style=\"color: #0000ff; text-decoration-color: #0000ff\">1</span>                                                                                    <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n","<span style=\"color: #800000; text-decoration-color: #800000\">│</span>                                                                                                  <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n","<span style=\"color: #800000; text-decoration-color: #800000\">│</span> <span style=\"color: #800000; text-decoration-color: #800000\">❱ </span>1 <span style=\"color: #0000ff; text-decoration-color: #0000ff\">import</span> <span style=\"color: #00ffff; text-decoration-color: #00ffff; text-decoration: underline\">evaluate</span>                                                                              <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n","<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">2 </span><span style=\"color: #0000ff; text-decoration-color: #0000ff\">from</span> <span style=\"color: #00ffff; text-decoration-color: #00ffff; text-decoration: underline\">nltk.tokenize</span> <span style=\"color: #0000ff; text-decoration-color: #0000ff\">import</span> sent_tokenize                                                      <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n","<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">3 #!pip install rouge_score</span>                                                                    <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n","<span style=\"color: #800000; text-decoration-color: #800000\">│</span>   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">4 #!pip install rouge</span>                                                                          <span style=\"color: #800000; text-decoration-color: #800000\">│</span>\n","<span style=\"color: #800000; text-decoration-color: #800000\">╰──────────────────────────────────────────────────────────────────────────────────────────────────╯</span>\n","<span style=\"color: #ff0000; text-decoration-color: #ff0000; font-weight: bold\">ModuleNotFoundError: </span>No module named <span style=\"color: #008000; text-decoration-color: #008000\">'evaluate'</span>\n","</pre>\n"],"text/plain":["\u001b[31m╭─\u001b[0m\u001b[31m──────────────────────────────\u001b[0m\u001b[31m \u001b[0m\u001b[1;31mTraceback \u001b[0m\u001b[1;2;31m(most recent call last)\u001b[0m\u001b[31m \u001b[0m\u001b[31m───────────────────────────────\u001b[0m\u001b[31m─╮\u001b[0m\n","\u001b[31m│\u001b[0m in \u001b[92m<module>\u001b[0m:\u001b[94m1\u001b[0m                                                                                    \u001b[31m│\u001b[0m\n","\u001b[31m│\u001b[0m                                                                                                  \u001b[31m│\u001b[0m\n","\u001b[31m│\u001b[0m \u001b[31m❱ \u001b[0m1 \u001b[94mimport\u001b[0m \u001b[4;96mevaluate\u001b[0m                                                                              \u001b[31m│\u001b[0m\n","\u001b[31m│\u001b[0m   \u001b[2m2 \u001b[0m\u001b[94mfrom\u001b[0m \u001b[4;96mnltk\u001b[0m\u001b[4;96m.\u001b[0m\u001b[4;96mtokenize\u001b[0m \u001b[94mimport\u001b[0m sent_tokenize                                                      \u001b[31m│\u001b[0m\n","\u001b[31m│\u001b[0m   \u001b[2m3 \u001b[0m\u001b[2m#!pip install rouge_score\u001b[0m                                                                    \u001b[31m│\u001b[0m\n","\u001b[31m│\u001b[0m   \u001b[2m4 \u001b[0m\u001b[2m#!pip install rouge\u001b[0m                                                                          \u001b[31m│\u001b[0m\n","\u001b[31m╰──────────────────────────────────────────────────────────────────────────────────────────────────╯\u001b[0m\n","\u001b[1;91mModuleNotFoundError: \u001b[0mNo module named \u001b[32m'evaluate'\u001b[0m\n"]},"metadata":{},"output_type":"display_data"}],"source":["import evaluate\n","from nltk.tokenize import sent_tokenize\n","#!pip install rouge_score\n","#!pip install rouge"]},{"cell_type":"code","execution_count":null,"id":"2d8739b9","metadata":{"execution":{"iopub.execute_input":"2023-08-14T15:44:35.250699Z","iopub.status.busy":"2023-08-14T15:44:35.25025Z","iopub.status.idle":"2023-08-14T15:44:37.821374Z","shell.execute_reply":"2023-08-14T15:44:37.820201Z","shell.execute_reply.started":"2023-08-14T15:44:35.250663Z"},"papermill":{"duration":null,"end_time":null,"exception":null,"start_time":null,"status":"pending"},"tags":[]},"outputs":[],"source":["import evaluate\n","from nltk.tokenize import sent_tokenize\n","#from rouge_score import rouge_scorer"]},{"cell_type":"code","execution_count":null,"id":"ffc7d23e","metadata":{"execution":{"iopub.execute_input":"2023-08-14T15:44:38.982027Z","iopub.status.busy":"2023-08-14T15:44:38.980575Z","iopub.status.idle":"2023-08-14T15:44:39.388636Z","shell.execute_reply":"2023-08-14T15:44:39.387254Z","shell.execute_reply.started":"2023-08-14T15:44:38.981986Z"},"papermill":{"duration":null,"end_time":null,"exception":null,"start_time":null,"status":"pending"},"tags":[]},"outputs":[],"source":["#With the function load of the library evaluate \n","#we create a rouge_score object\n","rouge_score = evaluate.load(\"rouge\")"]},{"cell_type":"markdown","id":"80e413b0","metadata":{"papermill":{"duration":null,"end_time":null,"exception":null,"start_time":null,"status":"pending"},"tags":[]},"source":["Calculating ROUGE is as simple as calling the *compute* function of the *rouge_score* object we created earlier. This function takes the texts to compare as arguments and a third value *use_stemmer*, which indicates whether it should use *stemmer* or full words for the comparison.\n","\n","A *stemmer* is the base of the word. Transform differents forms of a word in a same base. \n","\n","Some samples of steammer are: \n","* Jumping -> Jump. \n","* Running -> Run. \n","* Cats -> Cat. "]},{"cell_type":"code","execution_count":null,"id":"d42c5f2f","metadata":{"execution":{"iopub.execute_input":"2023-08-14T16:04:20.076601Z","iopub.status.busy":"2023-08-14T16:04:20.076189Z","iopub.status.idle":"2023-08-14T16:04:20.083634Z","shell.execute_reply":"2023-08-14T16:04:20.082378Z","shell.execute_reply.started":"2023-08-14T16:04:20.076571Z"},"papermill":{"duration":null,"end_time":null,"exception":null,"start_time":null,"status":"pending"},"tags":[]},"outputs":[],"source":["def compute_rouge_score(generated, reference):\n","    \n","    #We need to add '\\n' to each line before send it to ROUGE\n","    generated_with_newlines = [\"\\n\".join(sent_tokenize(s.strip())) for s in generated]\n","    reference_with_newlines = [\"\\n\".join(sent_tokenize(s.strip())) for s in reference]\n","    \n","    return rouge_score.compute(\n","        predictions=generated_with_newlines,\n","        references=reference_with_newlines,\n","        use_stemmer=True,\n","        \n","    )"]},{"cell_type":"code","execution_count":null,"id":"6eab715f","metadata":{"execution":{"iopub.execute_input":"2023-08-14T16:04:20.824031Z","iopub.status.busy":"2023-08-14T16:04:20.823653Z","iopub.status.idle":"2023-08-14T16:04:21.027311Z","shell.execute_reply":"2023-08-14T16:04:21.026243Z","shell.execute_reply.started":"2023-08-14T16:04:20.824001Z"},"papermill":{"duration":null,"end_time":null,"exception":null,"start_time":null,"status":"pending"},"tags":[]},"outputs":[],"source":["compute_rouge_score(summaries_small, summaries_reference)"]},{"cell_type":"markdown","id":"8a26c505","metadata":{"papermill":{"duration":null,"end_time":null,"exception":null,"start_time":null,"status":"pending"},"tags":[]},"source":["We can see that there is a difference between the two models when performing summarization. \n","\n","For example, in ROUGE-1, the similarity is 47%, while in ROUGE-2, it's a 32%. This indicates that the results are different, with some similarities but differents enough. \n","\n","However, we still don't know which model is better since we have compared them to each other and not to a reference text. But at the very least, we know that the fine-tuning process applied to the second model has significantly altered its results."]},{"cell_type":"markdown","id":"bb4d432c","metadata":{"execution":{"iopub.execute_input":"2023-08-07T17:21:28.296636Z","iopub.status.busy":"2023-08-07T17:21:28.296173Z","iopub.status.idle":"2023-08-07T17:21:28.301906Z","shell.execute_reply":"2023-08-07T17:21:28.300702Z","shell.execute_reply.started":"2023-08-07T17:21:28.296601Z"},"papermill":{"duration":null,"end_time":null,"exception":null,"start_time":null,"status":"pending"},"tags":[]},"source":["# Comparing to a Dataset with real summaries. \n","We are going to load the Dataset cnn_dailymail. This is a well-known dataset available in the **Datasets** library, and it suits our purpose perfectly. \n","\n","Apart from the news, it also contains pre-existing summaries. \n","\n","We will compare the summaries generated by the two models we are using with those from the dataset to determine which model creates summaries that are closer to the reference ones."]},{"cell_type":"code","execution_count":null,"id":"97babcd2","metadata":{"execution":{"iopub.execute_input":"2023-08-14T16:26:09.837146Z","iopub.status.busy":"2023-08-14T16:26:09.836684Z","iopub.status.idle":"2023-08-14T16:29:24.075679Z","shell.execute_reply":"2023-08-14T16:29:24.074315Z","shell.execute_reply.started":"2023-08-14T16:26:09.837112Z"},"papermill":{"duration":null,"end_time":null,"exception":null,"start_time":null,"status":"pending"},"tags":[]},"outputs":[],"source":["from datasets import load_dataset\n","\n","cnn_dataset = load_dataset(\n","    \"cnn_dailymail\", version=\"3.0.0\"\n",")\n","\n","#Get just a few news to test\n","sample_cnn = cnn_dataset[\"test\"].select(range(MAX_NEWS))\n","\n","sample_cnn"]},{"cell_type":"markdown","id":"be652a76","metadata":{"papermill":{"duration":null,"end_time":null,"exception":null,"start_time":null,"status":"pending"},"tags":[]},"source":["We retrieve the maximum length of the summaries to give the models the option to generate summaries of the same length, if they choose to do so."]},{"cell_type":"code","execution_count":null,"id":"e8546de4","metadata":{"execution":{"iopub.execute_input":"2023-08-14T16:29:24.079892Z","iopub.status.busy":"2023-08-14T16:29:24.078748Z","iopub.status.idle":"2023-08-14T16:29:24.0898Z","shell.execute_reply":"2023-08-14T16:29:24.085476Z","shell.execute_reply.started":"2023-08-14T16:29:24.079854Z"},"papermill":{"duration":null,"end_time":null,"exception":null,"start_time":null,"status":"pending"},"tags":[]},"outputs":[],"source":["max_length = max(len(item['highlights']) for item in sample_cnn)\n","max_length = max_length + 10"]},{"cell_type":"code","execution_count":null,"id":"9389d76f","metadata":{"execution":{"iopub.execute_input":"2023-08-14T16:29:24.093133Z","iopub.status.busy":"2023-08-14T16:29:24.091136Z","iopub.status.idle":"2023-08-14T16:29:46.747981Z","shell.execute_reply":"2023-08-14T16:29:46.746942Z","shell.execute_reply.started":"2023-08-14T16:29:24.093079Z"},"papermill":{"duration":null,"end_time":null,"exception":null,"start_time":null,"status":"pending"},"tags":[]},"outputs":[],"source":["summaries_t5_base = create_summaries(sample_cnn[\"article\"], \n","                                      tokenizer_small, \n","                                      model_small, \n","                                      max_l=max_length)"]},{"cell_type":"code","execution_count":null,"id":"67ab5985","metadata":{"execution":{"iopub.execute_input":"2023-08-14T16:29:46.751951Z","iopub.status.busy":"2023-08-14T16:29:46.750888Z","iopub.status.idle":"2023-08-14T16:30:05.204078Z","shell.execute_reply":"2023-08-14T16:30:05.203025Z","shell.execute_reply.started":"2023-08-14T16:29:46.7519Z"},"papermill":{"duration":null,"end_time":null,"exception":null,"start_time":null,"status":"pending"},"tags":[]},"outputs":[],"source":["summaries_t5_finetuned = create_summaries(sample_cnn[\"article\"], \n","                                      tokenizer_reference, \n","                                      model_reference, \n","                                      max_l=max_length)"]},{"cell_type":"code","execution_count":null,"id":"7b80c0c1","metadata":{"execution":{"iopub.execute_input":"2023-08-14T16:35:57.777001Z","iopub.status.busy":"2023-08-14T16:35:57.776036Z","iopub.status.idle":"2023-08-14T16:35:57.784049Z","shell.execute_reply":"2023-08-14T16:35:57.782555Z","shell.execute_reply.started":"2023-08-14T16:35:57.776958Z"},"papermill":{"duration":null,"end_time":null,"exception":null,"start_time":null,"status":"pending"},"tags":[]},"outputs":[],"source":["#Get the real summaries from the cnn_dataset\n","real_summaries = sample_cnn['highlights']"]},{"cell_type":"markdown","id":"a8dc3a10","metadata":{"papermill":{"duration":null,"end_time":null,"exception":null,"start_time":null,"status":"pending"},"tags":[]},"source":["Let's take a look at the generated summaries alongside the reference summaries provided by the dataset."]},{"cell_type":"code","execution_count":null,"id":"d6ea4e8c","metadata":{"execution":{"iopub.execute_input":"2023-08-14T16:36:00.561736Z","iopub.status.busy":"2023-08-14T16:36:00.561112Z","iopub.status.idle":"2023-08-14T16:36:00.577942Z","shell.execute_reply":"2023-08-14T16:36:00.576678Z","shell.execute_reply.started":"2023-08-14T16:36:00.561681Z"},"papermill":{"duration":null,"end_time":null,"exception":null,"start_time":null,"status":"pending"},"tags":[]},"outputs":[],"source":["summaries = pd.DataFrame.from_dict(\n","        {\n","            \"base\": summaries_t5_base, \n","            \"finetuned\": summaries_t5_finetuned,\n","            \"reference\": real_summaries,\n","        }\n","    )\n","summaries.head()"]},{"cell_type":"markdown","id":"c73e7277","metadata":{"papermill":{"duration":null,"end_time":null,"exception":null,"start_time":null,"status":"pending"},"tags":[]},"source":["Now we can calculate the ROUGE scores for the two models."]},{"cell_type":"code","execution_count":null,"id":"c0bda484","metadata":{"execution":{"iopub.execute_input":"2023-08-14T16:37:53.508688Z","iopub.status.busy":"2023-08-14T16:37:53.506732Z","iopub.status.idle":"2023-08-14T16:37:53.827647Z","shell.execute_reply":"2023-08-14T16:37:53.826103Z","shell.execute_reply.started":"2023-08-14T16:37:53.508613Z"},"papermill":{"duration":null,"end_time":null,"exception":null,"start_time":null,"status":"pending"},"tags":[]},"outputs":[],"source":["compute_rouge_score(summaries_t5_base, real_summaries)"]},{"cell_type":"code","execution_count":null,"id":"a4045072","metadata":{"execution":{"iopub.execute_input":"2023-08-14T16:37:55.512869Z","iopub.status.busy":"2023-08-14T16:37:55.512163Z","iopub.status.idle":"2023-08-14T16:37:55.822842Z","shell.execute_reply":"2023-08-14T16:37:55.821446Z","shell.execute_reply.started":"2023-08-14T16:37:55.512794Z"},"papermill":{"duration":null,"end_time":null,"exception":null,"start_time":null,"status":"pending"},"tags":[]},"outputs":[],"source":["compute_rouge_score(summaries_t5_finetuned, real_summaries)"]},{"cell_type":"markdown","id":"c91b3f50","metadata":{"papermill":{"duration":null,"end_time":null,"exception":null,"start_time":null,"status":"pending"},"tags":[]},"source":["With these results, I would say that the fine-tuned model performs slightly better than the T5-Base model. It consistently achieves higher ROUGE scores in all metrics except for LSUM, where the difference is minimal.\n","\n","Additionally, the ROUGE metrics are quite interpretable. \n","\n","LSUM indicates the percentage of the longest common subsequence, regardless of word order, in relation to the total length of the text. \n","\n","This can be a good indicator of overall similarity between texts. However, both models have very similar LSUM scores, and the fine-tuned model has better scores in other ROUGE metrics.\n","\n","Personally, I would lean towards the fine-tuned model, although the difference may not be very significant.\n","\n","## Continue learning\n","This notebook is part of a [course on large language models](https://github.com/peremartra/Large-Language-Model-Notebooks-Course) I'm working on and it's available on [GitHub](https://github.com/peremartra/Large-Language-Model-Notebooks-Course). You can see the other lessons and if you like it, don't forget to subscribe to receive notifications of new lessons.\n","\n","Other notebooks in the Large Language Models series: \n","https://www.kaggle.com/code/peremartramanonellas/use-a-vectorial-db-to-optimize-prompts-for-llms\n","https://www.kaggle.com/code/peremartramanonellas/ask-your-documents-with-langchain-vectordb-hf\n"]},{"cell_type":"markdown","id":"f89d35ed","metadata":{"papermill":{"duration":null,"end_time":null,"exception":null,"start_time":null,"status":"pending"},"tags":[]},"source":["### Feel Free to fork or edit the noteboook for you own convenience. Please consider ***UPVOTING IT***. It helps others to discover the notebook, and it encourages me to continue publishing."]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.10.12"},"papermill":{"default_parameters":{},"duration":150.75,"end_time":"2023-08-14T22:04:41.050595","environment_variables":{},"exception":true,"input_path":"__notebook__.ipynb","output_path":"__notebook__.ipynb","parameters":{},"start_time":"2023-08-14T22:02:10.300595","version":"2.4.0"},"widgets":{"application/vnd.jupyter.widget-state+json":{"state":{"0073e31caea541a2895d88e888e19fdb":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_23f8ceb6921143d28da83e2d9a4db301","placeholder":"​","style":"IPY_MODEL_9edb6408034242d095ef055cc0650515","value":" 1.92k/1.92k [00:00&lt;00:00, 127kB/s]"}},"0458785f3362447fb2613f72f3233c31":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"05d3848964184b689ad735f742287cd4":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"072a5425808a4c85ba8d48d544bbea0a":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"07ad9eb16e7b4809b1737b7163ee8087":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"0b3aea7574ec4faba22c1b9ece673adf":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_9235ee849b854b3ebbc3e82682ed296d","placeholder":"​","style":"IPY_MODEL_d08ca9940df245cea702d0c80a19ca15","value":" 1.21k/1.21k [00:00&lt;00:00, 89.3kB/s]"}},"0daa3e260d4244da9ff361bf9aff749d":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"0e8b9721f1df471491fb5ce5a8f8b701":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"0fb7a5a9dbb042a785be64a32c2b8d6f":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_8a26fe1d2b4d420ca6322b5704110a74","placeholder":"​","style":"IPY_MODEL_171a1ee8fed64a1ca669c6086e2fd790","value":" 1.79k/1.79k [00:00&lt;00:00, 137kB/s]"}},"10f7887954d64a43a34c0f2edd22f4ab":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"141e62e14bd04251b49f1a12dba527fb":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"15b7072970ed484eaac41012f4fae03c":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_1b6189642c074671bab5ae6afb6c3998","placeholder":"​","style":"IPY_MODEL_62b017a693e14a5ab2762c73745ef528","value":"Downloading model.safetensors: 100%"}},"171a1ee8fed64a1ca669c6086e2fd790":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"1910a41d404846d3a6a5adaf2945e51b":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"1b6189642c074671bab5ae6afb6c3998":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"1c6424e9bb1f44589a3b6153f7d8aaf1":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_15b7072970ed484eaac41012f4fae03c","IPY_MODEL_cce873e4ed1b48c39c81ed16fabe1adb","IPY_MODEL_8bce20f753104e95b4d2df3e67360dab"],"layout":"IPY_MODEL_d3dc2c82f6f44f80aebd023f9bbb40cb"}},"1ff74ca8cb38452b8145dda95757a0fb":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_8d3ce870642e4ba7a81ef422de0c5e33","IPY_MODEL_8db4d6cfda864d038b34a6c156dc91f7","IPY_MODEL_0fb7a5a9dbb042a785be64a32c2b8d6f"],"layout":"IPY_MODEL_d038e7d3a2214383a0a49df47833171d"}},"2082594e5d93416c9e1b8afacd1b96b3":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"215f72e30e3b4f7594f57fc9be2aa998":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"227ac036d7b04e7fbc905e2fc7f3f20f":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_697b5d0604d24fb2aaf04d21f934bb05","placeholder":"​","style":"IPY_MODEL_e54aa48d78b34c7c8a5ed1ad11a395bc","value":"Downloading (…)lve/main/config.json: 100%"}},"2347ea7631f540cfa98232053cdf2345":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"2360d48f25ef49e499d20c9728d7d40f":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"23f8ceb6921143d28da83e2d9a4db301":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"26341f68d3c84deba805ae56c8eba47a":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_7766a5dcf03a4212a187969540f0b0c2","placeholder":"​","style":"IPY_MODEL_5a5dcddec4e3446a9b50ef541f52c7ae","value":"Downloading (…)/main/tokenizer.json: 100%"}},"285c481ce8e84edda83a484f7008a11b":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"2b2463a361b24402ad7834c48ff13781":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_227ac036d7b04e7fbc905e2fc7f3f20f","IPY_MODEL_eb2adcd73a3245e7b461657b5734b1e3","IPY_MODEL_826553c8b96b4ea7bbaf6c87a1e4a87b"],"layout":"IPY_MODEL_5866e4ed799b4cf998bf040ef18b6957"}},"2cf2b2790bf442a2a5d4d40be7d044f3":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_7a46b31f28dc408ea624b7bf4d614362","placeholder":"​","style":"IPY_MODEL_849a233bac474b75abd36c309c5fbba9","value":"Downloading (…)lve/main/config.json: 100%"}},"33a099f8bc0048ecb1b9afafc5992396":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"343103993e6d446cabafade01de2a418":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_8fb944d0967a4bd9a89bd898f308c766","placeholder":"​","style":"IPY_MODEL_68ccc56ae3c148529d12e579822daeac","value":" 892M/892M [00:02&lt;00:00, 373MB/s]"}},"354cada5920547c3b88b11602912968b":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"367e3c7738464297a8eb568b17cd294b":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_b41e8871c28a475c8f5866dc8d81ccc0","placeholder":"​","style":"IPY_MODEL_2347ea7631f540cfa98232053cdf2345","value":"Downloading (…)/main/tokenizer.json: 100%"}},"3690c05242c24fb3a72ce132acdde995":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"3847a4017e094be794d7fbb39fc905cc":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_072a5425808a4c85ba8d48d544bbea0a","placeholder":"​","style":"IPY_MODEL_0e8b9721f1df471491fb5ce5a8f8b701","value":"Downloading (…)ve/main/spiece.model: 100%"}},"3993827aede741db8671678eefd1b573":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"3dc1ad6041db45b58274d538a0fbdd92":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_0458785f3362447fb2613f72f3233c31","max":147.0,"min":0.0,"orientation":"horizontal","style":"IPY_MODEL_2360d48f25ef49e499d20c9728d7d40f","value":147.0}},"4059ca9f0eec4583a4b254ce235e0159":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"40af9bd9a8a74c0a857437ffdbf56deb":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"4246a421c6204b91a0b3f3d8b24753a9":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_a0d45d50ec8647e38ee7d22c51e1253f","IPY_MODEL_964b2ad7b9fb40eca1afe09e488c7bf6","IPY_MODEL_343103993e6d446cabafade01de2a418"],"layout":"IPY_MODEL_c204348fdfa64716979b41719e55bcc8"}},"4b20dfdba30342078e55ec3aaba678f6":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_2cf2b2790bf442a2a5d4d40be7d044f3","IPY_MODEL_e8abc4588cf744779059f12ba11285ca","IPY_MODEL_0b3aea7574ec4faba22c1b9ece673adf"],"layout":"IPY_MODEL_3993827aede741db8671678eefd1b573"}},"4c4b065791d54410b4898e983e32e568":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_367e3c7738464297a8eb568b17cd294b","IPY_MODEL_e89fc95e248a4c91adcff6b53a9f92ac","IPY_MODEL_e28565b708a94c1d845de2179697fc44"],"layout":"IPY_MODEL_141e62e14bd04251b49f1a12dba527fb"}},"4dc8c2f51bf34d379d4199225c988282":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"4ddf5ca24f2045d5b06657665b9317b7":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"5777b3b56bf44c84b0e5d8ce12c8979f":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"5866e4ed799b4cf998bf040ef18b6957":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"5a5dcddec4e3446a9b50ef541f52c7ae":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"60663f8f04a4489fb29dd3848a05e4d7":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"620e861781314e23aa1212ff989e928d":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"62b017a693e14a5ab2762c73745ef528":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"68ccc56ae3c148529d12e579822daeac":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"697b5d0604d24fb2aaf04d21f934bb05":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"6a698120503f4815b1887257c26c0a46":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"6c8aca16f03b4f85bd431d3e701beca9":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_3847a4017e094be794d7fbb39fc905cc","IPY_MODEL_f5cbf94f988d4905a56a066efcf26969","IPY_MODEL_89dfcc4f90c84a7594d3e5050d1b2c13"],"layout":"IPY_MODEL_eec4741599a9403f8c25fcfe6c08f7fb"}},"6ca1675908a342888efa5e8be4b2d39f":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"6dcd63f5bcae4b19a9ae0bb033e31caf":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_b9673e2350f648319b24355c54112b74","placeholder":"​","style":"IPY_MODEL_60663f8f04a4489fb29dd3848a05e4d7","value":"Downloading (…)neration_config.json: 100%"}},"74b7dd742d414a7799f160669fb1527c":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"7766a5dcf03a4212a187969540f0b0c2":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"79939baa019545ce8bc2a3a7d829eaa9":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"7a46b31f28dc408ea624b7bf4d614362":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"826553c8b96b4ea7bbaf6c87a1e4a87b":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_8a25afb92c3e4990827c20bf26df86d3","placeholder":"​","style":"IPY_MODEL_b7f41f8092214c1ba9ddf4820d0868fe","value":" 1.36k/1.36k [00:00&lt;00:00, 61.4kB/s]"}},"849a233bac474b75abd36c309c5fbba9":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"89dfcc4f90c84a7594d3e5050d1b2c13":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_0daa3e260d4244da9ff361bf9aff749d","placeholder":"​","style":"IPY_MODEL_10f7887954d64a43a34c0f2edd22f4ab","value":" 792k/792k [00:00&lt;00:00, 1.14MB/s]"}},"8a25afb92c3e4990827c20bf26df86d3":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"8a26fe1d2b4d420ca6322b5704110a74":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"8bce20f753104e95b4d2df3e67360dab":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_620e861781314e23aa1212ff989e928d","placeholder":"​","style":"IPY_MODEL_215f72e30e3b4f7594f57fc9be2aa998","value":" 892M/892M [00:40&lt;00:00, 21.7MB/s]"}},"8d148a6312664127b421ed1d157d80d6":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"8d3ce870642e4ba7a81ef422de0c5e33":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_b2d00d21c3d24a6a9dd7b9888c75720f","placeholder":"​","style":"IPY_MODEL_d989c20b7dad42b9b97c2909fe190a8c","value":"Downloading (…)cial_tokens_map.json: 100%"}},"8d6d9b74e3384af0943c475ed8db27a4":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"8db4d6cfda864d038b34a6c156dc91f7":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_a2b76c4747be47568a15d0e786eb865e","max":1786.0,"min":0.0,"orientation":"horizontal","style":"IPY_MODEL_285c481ce8e84edda83a484f7008a11b","value":1786.0}},"8fb944d0967a4bd9a89bd898f308c766":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"9235ee849b854b3ebbc3e82682ed296d":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"964b2ad7b9fb40eca1afe09e488c7bf6":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_4059ca9f0eec4583a4b254ce235e0159","max":891646390.0,"min":0.0,"orientation":"horizontal","style":"IPY_MODEL_3690c05242c24fb3a72ce132acdde995","value":891646390.0}},"994a0770e00545b2b9ace74b70dc0d2f":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"9cd2db54e977486d9c1b0f5ef5032a30":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_c7bb93c759f24438a97666eefe54f4e0","IPY_MODEL_a50df1215ece40c4bba5c5c64e4f26c3","IPY_MODEL_0073e31caea541a2895d88e888e19fdb"],"layout":"IPY_MODEL_d7a875bd377b4119ba2f8595e637d01a"}},"9edb6408034242d095ef055cc0650515":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"9f0af4b71f4f4ca6b42dbbfc699db7c2":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"a0d45d50ec8647e38ee7d22c51e1253f":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_df9a1242731a4af591807b40f9491e3f","placeholder":"​","style":"IPY_MODEL_5777b3b56bf44c84b0e5d8ce12c8979f","value":"Downloading model.safetensors: 100%"}},"a2b76c4747be47568a15d0e786eb865e":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"a2e7095f26a344bd949dc98fb51634fd":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"a50df1215ece40c4bba5c5c64e4f26c3":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_dca81b6f5cec4a988c1fa3cf1addcb76","max":1923.0,"min":0.0,"orientation":"horizontal","style":"IPY_MODEL_33a099f8bc0048ecb1b9afafc5992396","value":1923.0}},"afd407804c024cb6b1029e8ebad70f73":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_6dcd63f5bcae4b19a9ae0bb033e31caf","IPY_MODEL_3dc1ad6041db45b58274d538a0fbdd92","IPY_MODEL_c94d12dd3a7745e5b91fb0535094c63a"],"layout":"IPY_MODEL_74b7dd742d414a7799f160669fb1527c"}},"b2d00d21c3d24a6a9dd7b9888c75720f":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"b41e8871c28a475c8f5866dc8d81ccc0":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"b7f41f8092214c1ba9ddf4820d0868fe":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"b9673e2350f648319b24355c54112b74":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"bfee127e736f42cb97c4010d0d21d447":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_4dc8c2f51bf34d379d4199225c988282","max":1389353.0,"min":0.0,"orientation":"horizontal","style":"IPY_MODEL_79939baa019545ce8bc2a3a7d829eaa9","value":1389353.0}},"c204348fdfa64716979b41719e55bcc8":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"c3c506c776fa4649aeb00ed0d6096dff":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"c7bb93c759f24438a97666eefe54f4e0":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_6a698120503f4815b1887257c26c0a46","placeholder":"​","style":"IPY_MODEL_6ca1675908a342888efa5e8be4b2d39f","value":"Downloading (…)okenizer_config.json: 100%"}},"c94d12dd3a7745e5b91fb0535094c63a":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_c3c506c776fa4649aeb00ed0d6096dff","placeholder":"​","style":"IPY_MODEL_8d6d9b74e3384af0943c475ed8db27a4","value":" 147/147 [00:00&lt;00:00, 11.4kB/s]"}},"cce873e4ed1b48c39c81ed16fabe1adb":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_40af9bd9a8a74c0a857437ffdbf56deb","max":891558694.0,"min":0.0,"orientation":"horizontal","style":"IPY_MODEL_db71fe89c44d4806bc4074df116125c8","value":891558694.0}},"d038e7d3a2214383a0a49df47833171d":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"d08ca9940df245cea702d0c80a19ca15":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"d3dc2c82f6f44f80aebd023f9bbb40cb":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"d5e5556ba450419687025020eeb37d54":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"d7a875bd377b4119ba2f8595e637d01a":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"d989c20b7dad42b9b97c2909fe190a8c":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"db71fe89c44d4806bc4074df116125c8":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"dca81b6f5cec4a988c1fa3cf1addcb76":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"df9a1242731a4af591807b40f9491e3f":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"e05ee0ecd2934c4e8d14e4d8b757a71f":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"e28565b708a94c1d845de2179697fc44":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_e7c70483480042c9bd839a0a283d6ec7","placeholder":"​","style":"IPY_MODEL_05d3848964184b689ad735f742287cd4","value":" 1.39M/1.39M [00:00&lt;00:00, 7.70MB/s]"}},"e379da4133754a338243c8dfeb7f3087":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_1910a41d404846d3a6a5adaf2945e51b","placeholder":"​","style":"IPY_MODEL_9f0af4b71f4f4ca6b42dbbfc699db7c2","value":" 1.39M/1.39M [00:00&lt;00:00, 1.62MB/s]"}},"e54aa48d78b34c7c8a5ed1ad11a395bc":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"e7c70483480042c9bd839a0a283d6ec7":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"e89fc95e248a4c91adcff6b53a9f92ac":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_2082594e5d93416c9e1b8afacd1b96b3","max":1387438.0,"min":0.0,"orientation":"horizontal","style":"IPY_MODEL_4ddf5ca24f2045d5b06657665b9317b7","value":1387438.0}},"e8abc4588cf744779059f12ba11285ca":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_994a0770e00545b2b9ace74b70dc0d2f","max":1208.0,"min":0.0,"orientation":"horizontal","style":"IPY_MODEL_07ad9eb16e7b4809b1737b7163ee8087","value":1208.0}},"eb2adcd73a3245e7b461657b5734b1e3":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_d5e5556ba450419687025020eeb37d54","max":1363.0,"min":0.0,"orientation":"horizontal","style":"IPY_MODEL_8d148a6312664127b421ed1d157d80d6","value":1363.0}},"eec4741599a9403f8c25fcfe6c08f7fb":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"f5cbf94f988d4905a56a066efcf26969":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_354cada5920547c3b88b11602912968b","max":791656.0,"min":0.0,"orientation":"horizontal","style":"IPY_MODEL_a2e7095f26a344bd949dc98fb51634fd","value":791656.0}},"f7fb4e3a04be4b31a36924431df8f33a":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_26341f68d3c84deba805ae56c8eba47a","IPY_MODEL_bfee127e736f42cb97c4010d0d21d447","IPY_MODEL_e379da4133754a338243c8dfeb7f3087"],"layout":"IPY_MODEL_e05ee0ecd2934c4e8d14e4d8b757a71f"}}},"version_major":2,"version_minor":0}}},"nbformat":4,"nbformat_minor":5}